{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sebastian Raschka, 2015"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Training a model for movie review classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from nltk.stem.porter import PorterStemmer\n",
    "import re\n",
    "from nltk.corpus import stopwords\n",
    "\n",
    "stop = stopwords.words('english')\n",
    "porter = PorterStemmer()\n",
    "\n",
    "def tokenizer(text):\n",
    "    text = re.sub('<[^>]*>', '', text)\n",
    "    emoticons = re.findall('(?::|;|=)(?:-)?(?:\\)|\\(|D|P)', text.lower())\n",
    "    text = re.sub('[\\W]+', ' ', text.lower()) + ' '.join(emoticons).replace('-', '')\n",
    "    text = [w for w in text.split() if w not in stop]\n",
    "    tokenized = [porter.stem(w) for w in text]\n",
    "    return tokenized\n",
    "\n",
    "def stream_docs(path):\n",
    "    with open(path, 'r') as csv:\n",
    "        next(csv) # skip header\n",
    "        for line in csv:\n",
    "            text, label = line[:-3], int(line[-2])\n",
    "            yield text, label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('\"In 1974, the teenager Martha Moxley (Maggie Grace) moves to the high-class area of Belle Haven, Greenwich, Connecticut. On the Mischief Night, eve of Halloween, she was murdered in the backyard of her house and her murder remained unsolved. Twenty-two years later, the writer Mark Fuhrman (Christopher Meloni), who is a former LA detective that has fallen in disgrace for perjury in O.J. Simpson trial and moved to Idaho, decides to investigate the case with his partner Stephen Weeks (Andrew Mitchell) with the purpose of writing a book. The locals squirm and do not welcome them, but with the support of the retired detective Steve Carroll (Robert Forster) that was in charge of the investigation in the 70\\'s, they discover the criminal and a net of power and money to cover the murder.<br /><br />\"\"Murder in Greenwich\"\" is a good TV movie, with the true story of a murder of a fifteen years old girl that was committed by a wealthy teenager whose mother was a Kennedy. The powerful and rich family used their influence to cover the murder for more than twenty years. However, a snoopy detective and convicted perjurer in disgrace was able to disclose how the hideous crime was committed. The screenplay shows the investigation of Mark and the last days of Martha in parallel, but there is a lack of the emotion in the dramatization. My vote is seven.<br /><br />Title (Brazil): Not Available\"',\n",
       " 1)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(stream_docs(path='../../data/movie_data.csv'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def get_minibatch(doc_stream, size):\n",
    "    docs, y = [], []\n",
    "    for _ in range(size):\n",
    "        text, label = next(doc_stream)\n",
    "        docs.append(text)\n",
    "        y.append(label)\n",
    "    return docs, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import HashingVectorizer\n",
    "from sklearn.linear_model import SGDClassifier\n",
    "\n",
    "vect = HashingVectorizer(decode_error='ignore', \n",
    "                         n_features=2**21,\n",
    "                         preprocessor=None, \n",
    "                         tokenizer=tokenizer)\n",
    "\n",
    "clf = SGDClassifier(loss='log', random_state=1, n_iter=1)\n",
    "doc_stream = stream_docs(path='../../data/movie_data.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "0%                          100%\n",
      "[##############################] | ETA[sec]: 0.000 \n",
      "Total time elapsed: 163.780 sec\n"
     ]
    }
   ],
   "source": [
    "import pyprind\n",
    "pbar = pyprind.ProgBar(45)\n",
    "\n",
    "classes = np.array([0, 1])\n",
    "for _ in range(45):\n",
    "    X_train, y_train = get_minibatch(doc_stream, size=1000)\n",
    "    X_train = vect.transform(X_train)\n",
    "    clf.partial_fit(X_train, y_train, classes=classes)\n",
    "    pbar.update()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.873\n"
     ]
    }
   ],
   "source": [
    "X_test, y_test = get_minibatch(doc_stream, size=5000)\n",
    "X_test = vect.transform(X_test)\n",
    "print('Accuracy: %.3f' % clf.score(X_test, y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "clf = clf.partial_fit(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Serializing fitted scikit-learn estimators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After we trained the logistic regression model as shown above, we know save the classifier along woth the stop words, Porter Stemmer, and `HashingVectorizer` as serialized objects to our local disk so that we can use the fitted classifier in our web application later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "import os\n",
    "\n",
    "dest = os.path.join('movieclassifier', 'pkl_objects')\n",
    "if not os.path.exists(dest):\n",
    "    os.mkdir(dest)\n",
    "\n",
    "pickle.dump(stop, open(os.path.join(dest, 'stopwords.pkl'), 'wb'))\n",
    "pickle.dump(porter, open(os.path.join(dest, 'porterstemmer.pkl'), 'wb'))    \n",
    "pickle.dump(vect, open(os.path.join(dest, 'vectorizer.pkl'), 'wb'))\n",
    "pickle.dump(clf, open(os.path.join(dest, 'classifier.pkl'), 'wb'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After executing the preceeding code cells, we can now restart the IPython notebook kernel to check if the objects were serialized correctly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pickle\n",
    "import re\n",
    "import os\n",
    "\n",
    "def tokenizer(text):\n",
    "    text = re.sub('<[^>]*>', '', text)\n",
    "    emoticons = re.findall('(?::|;|=)(?:-)?(?:\\)|\\(|D|P)', text.lower())\n",
    "    text = re.sub('[\\W]+', ' ', text.lower()) + ' '.join(emoticons).replace('-', '')\n",
    "    text = [w for w in text.split() if w not in stop]\n",
    "    tokenized = [porter.stem(w) for w in text]\n",
    "    return text\n",
    "\n",
    "dest = os.path.join('movieclassifier', 'pkl_objects')\n",
    "\n",
    "stop = pickle.load(open(os.path.join(dest, 'stopwords.pkl'), 'rb'))\n",
    "porter = pickle.load(open(os.path.join(dest, 'porterstemmer.pkl'), 'rb'))\n",
    "vect = pickle.load(open(os.path.join(dest, 'vectorizer.pkl'), 'rb'))\n",
    "clf = pickle.load(open(os.path.join(dest, 'classifier.pkl'), 'rb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prediction: positive\n",
      "Probability: 91.56%\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "label = {0:'negative', 1:'positive'}\n",
    "\n",
    "example = ['I love this movie']\n",
    "X = vect.transform(example)\n",
    "print('Prediction: %s\\nProbability: %.2f%%' %\\\n",
    "      (label[clf.predict(X)[0]], np.max(clf.predict_proba(X))*100))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating an SQLite Database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import sqlite3\n",
    "import os\n",
    "dest = os.path.join('movieclassifier', 'reviews.sqlite')\n",
    "conn = sqlite3.connect(dest)\n",
    "c = conn.cursor()\n",
    "c.execute('CREATE TABLE review_db (review TEXT, sentiment INTEGER, date TEXT)')\n",
    "\n",
    "example1 = 'I love this movie'\n",
    "c.execute(\"INSERT INTO review_db (review, sentiment, date) VALUES (?, ?, DATETIME('now'))\", (example1, 1))\n",
    "\n",
    "example2 = 'I disliked this movie'\n",
    "c.execute(\"INSERT INTO review_db (review, sentiment, date) VALUES (?, ?, DATETIME('now'))\", (example2, 0))\n",
    "\n",
    "conn.commit()\n",
    "conn.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "conn = sqlite3.connect(dest)\n",
    "c = conn.cursor()\n",
    "\n",
    "c.execute(\"SELECT * FROM review_db WHERE date BETWEEN '2015-01-01 10:10:10' AND DATETIME('now')\")\n",
    "results = c.fetchall()\n",
    "\n",
    "conn.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[('I love this movie', 1, '2015-06-02 16:02:12'), ('I disliked this movie', 0, '2015-06-02 16:02:12')]\n"
     ]
    }
   ],
   "source": [
    "print(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
